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ABSTRACT 

In recent work, we studied the problem of causally reconstructing 
time sequences of spatially sparse signals, with unknown and slow 
time- varying sparsity patterns, from a limited number of linear "in- 
coherent" measurements. We proposed a solution called Kalman 
Filtered Compressed Sensing (KF-CS). The key idea is to run a re- 
duced order KF only for the current signal's estimated nonzero co- 
efficients' set, while performing CS on the Kalman filtering error to 
estimate new additions, if any, to the set. KF may be replaced by 
Least Squares (LS) estimation and we call the resulting algorithm 
LS-CS. In this work, (a) we bound the error in performing CS on the 
LS error and (b) we obtain the conditions under which the KF-CS 
(or LS-CS) estimate converges to that of a genie-aided KF (or LS), 
i.e. the KF (or LS) which knows the true nonzero sets. 
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1. INTRODUCTION 

In recent work |T|, we studied the problem of causally reconstruct- 
ing time sequences of sparse signals, with unknown and slow time- 
varying sparsity patterns, from a limited number of noise-con'upted 
"incoherent" measurements. We proposed a solution called Kalman 
Filtered Compressed Sensing (KF-CS). With the exception of CS ID 
and of fSl, most other work |4l[5| treats the entire time sequence of 
signals/images as a single spatiotemporal signal and performs CS to 
reconstruct it. This is a non-causal solution and also has high com- 
putational cost. On the other hand, if the number of observations 
is small, performing CS [2] at each time (simple CS) incurs much 
larger error than KF-CS, see Fig. [T] Potential applications of KF- 
CS include making dynamic MRI real-time (causal and fast enough) 
(4l[6l; real-time video imaging using a single-pixel camera Q; or 
real-time tracking of temperature, or other, time-varying fields using 
sensor networks that transmit random projections of the field |7). 

In this work, in Sec. (2] we describe a simple modification of 
KF-CS im and introduce its non-Bayesian version. Least Squares 
(LS)-CS. Our key contributions are: (a) in Sec. [S] we bound the 
error in performing CS on the LS error in the observation and com- 
pare it with that for performing CS on the observation (simple CS), 
and (b) in Sec. |4l we obtain the conditions under which the KF-CS 
(or LS-CS) estimate converges to that of a genie-aided KF (or LS). 
Simulation comparisons are given in Sec. [5] 

Problem Definition. The problem definition is the same as in 
m . Let (zt)mxi denote the spatial signal of interest at time t and 
{yt)nx\, with n < m, denote its noise-corrupted observation vector 
at t. The signal, zt, is sparse in a given sparsity basis (e.g. wavelet) 
with orthonormal basis matrix, "3?mxm, i.e. Xt = zt is a sparse 
vector (only St « m elements of xt are non-zero). The observa- 
tions are "incoherent" w.r.t. the sparsity basis of the signal, i.e. 

yt = Axt+wt, H<^, E[wt\ = 0, ¥.[wtw[\ = al^J (1) 
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and all St-column sub-matrices of A "approximately orthonormal", 
i.e. Sst < 1 (H eq. (1.3)]. wt is independent of Xt and is i.i.d, Vt. 

Let Nt denote the the support set of xt, i.e. the set of its non-zero 
coordinates and let Tt = Nt denote its estimate. Also, let At denote 
the undetected nonzero set at time t, i.e. At = Nt\ Tt-\ and let At 
denote its estimate. Thus Tt = Tt-i U At. Let St = \Nt\ where 
|.| denotes the size of a set. Also, for any set T, let {v)t denote the 
\T\ length sub- vector containing the elements of v corresponding to 
the indices in the set T. For a matrix A, At denotes the sub-matrix 
obtained by extracting the columns of A corresponding to the indices 
in T. We use the notation {Q)ti,T2 to denote the sub-matrix of Q 
containing rows and columns corresponding to the entries in Ti and 
T2 respectively. T'^ denotes the complement of T w.r.t. [1 : m], i.e. 
T"^ = [1 : m]\T. (j> refers to the empty set. ' denotes transpose. 
The m X m matrix It is defined as follows: (It)t,t = I where I 
is a \T\-identity matrix while {It)t': ,[i:m] ~ 0, (/T)[i:m],T<: ~ 0. 

The nonzero coefficients' set A^t changes slowly over time. For 
the currently nonzero coefficients of Xt, {xt)Nt, we assume a spa- 
tially i.i.d. Gaussian random walk model, while the rest of the coef- 
ficients remain constant, i.e. 

Xo = 0, Xt = Xt-l + Vt, Ut ^ U{Q,Qt), Qt = UsysiNt (2) 

where ut is temporally i.i.d.. The current nonzero set, Nt, is un- 
known Vt. Our goal is to recursively get the best estimates of Nt and 
Xt (or equivalently of the signal, zt = ^xt) using yi, . . .yt- 

2. KALMAN FILTERED CS AND LEAST SQUARES CS 

We describe a simple modification of KF-CS (T| and introduce Least 
Squares CS. Let Xt\t-i, xt, Kt and Pt|t-ij Pt denote the predicted 
and updated state estimates at time t, the Kalman gain and the pre- 
diction and updated error covariances given by the KF in KF-CS 
(since KF-CS does not always use the correct value of Qt, Pt]t-i or 
Pt are not equal to the actual covariances of xt — Xt\t-i or Xt — xt). 

2.1. Modified Kalman Filtered Compressed Sensing (KF-CS) 

KF-CS can be summarized as running a KF for the system in (TJ, ([2} 
but with Qt replaced by Qt = cr^yslrt- The new additions, if any, 
are estimated by performing CS on the Kalman filtering error, yt.res- 
At time t, we first run a "temporary" Kalman prediction and 
update step using Qt = cr^yslrt^i, i.e. we compute 

Kt,tmp = (Pt-1 + Qt)A'{A{Pt^^ + Qt)A' + (jL/)"' 

Xt,tmp = [I — Kt,tmpA)xt-l + Kt,tmp yt (3) 

Let r = Tt-i. The filtering error is 

yt,res — yt — Axt,tmp = ^At (xt) At + Arixt — Xt)T + Wt (4) 

As explained in (2, if the filtering error norm is large, there is a need 
to estimate At. One can rewrite yt,tes as yt,tes — Afit + wt, where 
Pt = [(a;t — it)T, (a;t)At, 0(TuAt)<:l is a "sparse-compressible" 



Algorithm 1 Kalman Filtered Compressive Sensing (KF-CS) 



Initialization: Set xq = 0, Pq = 0, Tq = <j) (if unknown) or equal to tlie known support. For t > 0, do, 

1. Temporary Kalman prediction and update. Implement ^ using Qt = crsyslTt_i. 

2. Compute Additions using CS. Compute the KF error, yt jci — yt — Axt,tmp- Check if FEN = j/t.res^/e tJ/t.res > ci/e- If it is, 

(a) Do CS on yt.,es followed by thresholding, i.e. compute At using l|5j, l|6j. The new estimated support is Tt = Tt-i U At. 

3. Kalman prediction and update. Implement using Qt = olyslTt ■ 

(a) (KF-CS with final LS): If Tt / Tt^i, implement (O using Qt = oo/Tt, i.e. set xt = {A'^^Arty^ A'^^yt and {Pt)Tt,Tt = 

{A'T^AT.r^al,,, {Pt)T^, = 0, {Pt):,T^ = 0. 

4. Compute Deletions. If Tt == Tt_i ■ • ■ == Tt-k (nonzero set has not changed for long enough, i.e. w.h.p. KF stabilized), 

(a) Check for "zero" coefficients, i.e. compute Az — {i £ Tt : X]t=t-fe'+i(-'^^>0^/'^' < "^z} k' < k. Set Tt ^ T \ Az- 
Set(ft)A, =O.Set(Pt)A„[i:H =Oand(Pt)[i^„,,A, =0. " 

5. Output Tt, Xt and the signal estimate, it = $if . Increment t and go to the first step. 



signal with a "large" or "non-compressible" nonzero part, {xt)At, 
and a "small" or "compressible" nonzero part, {xt — Xt)T- The 
Dantzig selector (DS) followed by thresholding can be applied to 
detect the "non-compressible" nonzero part as follows: 

/3f = argmin s.t. \\A'{yt,mi ~ A(i)\\oo < \m(Toba (5) 

At = {i G TU ■ > q4 (6) 

where Am — \/2 log m and aa is the addition threshold. Thus, the 
estimated support set at time t is Tt = T U At = Tt^i U At. 

Next we run the Kalman prediction/update using Qt — crlyslrt '■ 

Pt\t-i = Pt-i + Qt, Kt = Pt|t_i^'(^Pt|t-iA' + a'tt,,!)-^ 
Pt = {I- KtA)Pt^t-i 

xt = {I~KtA)xt^i + Ktyt (7) 

with initialization Po = 0[i. , xo = 0[i.„]. 

Remark 1 For easy notation, in 0, write the KF equations 
for the entire xt- But actually we are running a reduced order KF 
for only the coefficients in T (T = Tt-i for ^ and T = Tt for |7|. 

2.1.1. Deleting Zero Coefficients 

If the addition threshold, aa , is not large enough, occasionally there 
will be some false additions (coefficients whose true value is zero 
but they wrongly get added due to error in the CS step). Also, there 
may be coefficients that actually become and remain zero. All such 
coefficients need to be detected and removed from Tt to prevent un- 
necessary increase in |Tt|. Increased |Tt| implies smaller minimum 
eigenvalue of A'j^^ Axt and thus increased estimation error. The in- 
crease is especially large if A'j^^ Art is close to becoming singular. 

One possible way to detect if a coefficient, i, is zero is to check if 
the magnitude of its estimates in the last few time instants is small, 
e.g. one can check if < a^. This scheme 

would be fairly accurate (small enough false alarm and miss proba- 
bilities), if the estimation error, Ct,! = XT,i — Xr,i is small enough, 
for all r G [t — k' + l,f\. If we check for zeroing only when T 
has not changed for long enough (w.h.p. this implies that all past 
additions have been detected, i.e. T = A'^t, and the KF for T 
has stabilized), the variance of er,i would be approximately equal 
to {Pt)i,i < a^tjJXminiA'rpAr), i.e. it would be small enough. 



When a coefficient, i, is detected as being zero, we remove it 

from Tt, we set it, i = and we set (Pt)i,[i;m] = 0, {Pt)iiym.],i = 0. 
We summarize the entire KF-CS algorithm in Algorithm|T] 

2.2. Least Squares CS: Non-Bayesian KF-CS 

In applications where training data is not be available to learn the 
prior model parameters required by KF-CS, one can use a non- 
Bayesian version of KF-CS i.e. replace the KF in KF-CS by Least 
Squares (LS) estimation. The LS step is also faster than the KF step. 

3. ANALYZING CS ON LS ERROR (LSE) 

Let T = Tt_i and A = At = A't \ Tt_i. The true nonzero sets 
at any time, A^t, are assumed to be non-random. But T = T-i 
is a random variable since its value depends on yt-i and Tt_2 (or 
equivalently on j/i;t-i). We use E[-] to denote expectation w.r.t. all 
random quantities (yi:t,xi;t at time t) while using E[-|j/i:t-i] to 
denote the expected value conditioned on yi:t-i. Conditioned on 
yi:t-i, the set T, and hence also the set A = A^t \ T, is known. 

The key difference between simple CS and LS-CS is that simple 
CS applies ^ on yt = Axt + Wt to estimate the |A''t| -sparse signal, 
Xt, while LS-CS applies ^ on the LS error (LSE), yt,xs '■= yt — 
Axt.tmp = ^/St+iut to estimate /3t := xt — xt.tmp, where ft, tmp = 
{A'j,ATy^A'j.yt. I3t = [{xt - it,tmp)T, (a;t)A, Otua<=] = 
{A'TATy^A'j.{AA{xt)A + 'wt),{xt)A,OTuA'^] is what we call 
a "sparse-compressible" signal; it is |T U A|-sparse but, if the spar- 
sity pattern changes slowly enough, it is compressible along T. We 
use this idea to bound the error in CS on LSE and to show that if the 
sparsity pattern changes slowly enough, the CS-LSE error bound is 
much smaller than that of simple CS. 

We use the following definition of compressibility of the random 
process /3t = (it{xt,yi:t). 

Definition 1 We say that (it is compressible if the maximum over 
T of the average of {Pt)i, conditioned on past observations, is 
smaller than the minimum average squared value of any cur- 
rently nonzero component of xt, i.e. i/maxigT E[(/3t)i |i/i:t-i] < 
minigiVt E[(a;t)i]. This is a valid definition since mini^Nt ^[(st)?] < 
minigA E[(xt)f] = min^eA E[(/?t)i]/or aW c/io;ce^ A = A(yi;t_i). 

Assumption 1 (model, algorithm) Assume that 

1. yt, Xt follow Q, (O; wt, ft are independent of each other 
and over time; and Wt has bounded support (e.g. truncated 
Gaussian ) with cutoffs at i: — ^'"Z°a''ii in all dimensions. 



2. iVt_i C Nt for all t and St := |iVt| < S^a.. 

3. The number of false additions is bounded, i.e. \Tt\Nt\ < Sfa 
for all t. This implies that \Tt \ < St + Sfa < Smax + Sfa- 

4- Ssrnai+Sja < 1- 5s = 5s{A) is defined in ^ eq. (1.3)]. 
Bounded measurement noise (Assumption ! II It is usually valid. As- 
sumption [Tl3] is observed in all our simulations, as long as the ad- 
dition threshold aa is large enough. Assumption 1 1 141 quantifies the 
required amount of incoherency of the measurement matrix w.r.t. the 
sparsity basis. Consider Assumption 1 1121 While this assumption is 
not strictly true, it is observed (for medical image sequences) that it 
is approximately true: the set Nt \ Nt~i, and the total set Nt \ No, 
are both small. Also, if we relax the definition of support to denote 
any set containing all nonzero elements of Xt, then this is true. 

Under the above assumptions, we can prove the following |S| : 

Theorem 1 Assume that Assumption\I]holds. Lett a = ta(t) denote 
the last addition time before or at t. 



1. If I A| is small enough to ensure that {t — ta 
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, then Pt '■= 



Xt — xt.tmp is compressible. 9s^s' " defined in ^ eq. ( 1.5)]. 
The following bound on the CS-LSE error holds 
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where xt,csLSE is the output of Q with yt,re. 
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and Xt,tmp = {A'-pAr) ^A'j,yt. 



NoticethatE[(xt)A(x-t)Aiyi:t-i], audits trace, E[|j (a::t)A|P|yi:t-i 
can be computed by running a genie-aided KF. 

In (Sj, we also derive a bound on the unconditional CS-LSE 
error, i.e. the error averaged over all values of the past observations, 
under slightly stronger assumptions. We also discuss why the bound 
on the CS-LSE error is much smaller than that on simple CS error. 

4. CONVERGENCE TO GENIE-AIDED KF (OR LS) 

Consider the genie-aided KF, i.e. the KF which knows the true 
nonzero set, Nt, at each t. It is the linear MMSE estimator of xt 
from yi, . . .yt if the nonzero sets, Nt 's, are known. It would be 
the MMSE estimator (i.e. it would be the best estimator among all 
possible estimators) if the observation noise were Gaussian instead 
of truncated Gaussian. The genie-aided KF can be summarized as 
running with Qt — cyaysiNf In this section, we obtain condi- 
tions under which the KF-CS estimate converges to the genie-aided 
KF estimate in probability. As a corollary, we also get conditions for 
LS-CS to converge to genie-aided LS. 

We begin by giving Lemma [T] states that if the true nonzero 
set does not change after a certain time, and if eventually it is cor- 
rectly detected, then KF-CS converges to GA-KF. This is followed 
by Lemmas |2] and [3] which prove that, if the addition threshold is 
high enough, the probability of false addition is zero and the prob- 
ability of correct set detection approaches one with t. Combining 
these lemmas gives the final result. 



Lemma 1 HEjj Assume that there exists a to s.t. V t > to, Tt — 
Nt = A''* and assume that S\ff^\ < 1. Consider KF-CS without the 
deletion step, i.e. with Uz = 0, and with the sten\3a\(KF-CS with 
final LS) replacing step\3\ifTt 7^ Tt-i. The difference in the KF-CS 
and GA-KF estimates, dt = \xt,GAKF — Xt\, converges to zero in 
mean square and hence also in probability. 

Assume that Assumption|T]holds. The bounded support assump- 
tion on wt ensures that \A'iWt\ < ||TOt||oo||^i||i < Xmt^obs, Vi. 
With this, the theorems of |2| can be directly modified to hold with 
probability one. This helps prove the following. 

Lemma 2 Assume that (i) Assumption\l\holds and that 52Smax + 
S'iSmax < 1 (stronger incoherency requirement than earlier); and 
(ii) in Algorithm\l\ we set ota = B\ = ClX^Smaxcy^hs (Ci is 
defined in ^ Thm. 1.1]). Then, at each t, the following hold: 

\\xt — Xt,tmp — < Bl = CiX^SmaxCobB (8) 

At C A^t, and so Tt (Z N, and so Tt U At+i = Nt+i (9) 

Proof: We prove this result by induction. At any t, when solv- 
ing (O, xt,tmp.i — 0, Wi £ Tt-i- The sparse vector to be es- 
timated is /3i = [{xt — xt,tmp)Tt_i,{xt)At,ON^]- First Consider 
the base case, t = 1. At t = 1, T-i = To = (f> (empty) and 
so xi^tmp.i = 0, Vi. Thus /3i = xi with nonzero set Ai = A^i. 
Since |A'^i| < Smax and since the observation noise, Wt, satisfies 
l^i™*! < AmXTobs, we can apply Theorem 1.1 of (2) to get ^ to 
always hold at f = L 

Also, for any i £ A''^ , xi^i — and so /3i — {xi.i — S 
lla^i — xi^tmp — /3i|P < Bl (from®). But = Bi. Thus, from 
if S> \A\ ^ ^ jYi. Thus Ti = To U Ai C A^i. But Ni C N2. Thus, 
Ti C N2. Since Ai = A^i \ To, this implies that Ti U A2 = N2. 
Thus ([9} also holds for t = 1. This proves the base case. 
if S < \ A\ For the inductive step, assume that ^ and ^ hold for f - 1. 

Thus, Tt-i U At = Nt, which is the nonzero set for /3t. But |A''t | < 
Smax. Thus Theorem 1.1 of |2| can be applied to get ([8} to hold for 
t. Also, for any i G A^t^^, xtA — and so /3t^, = (xt,i — Pt.iY ^ 
]^ \\xt — Xt,tmp — AlP < -Bl = aa. Thus from At C Ai't. Thus 
Tt = Tt-i U At C TVt C Nt+i. Since At+i = A^t+i \ T, this 
means that Tt U At+i = Nt+i. Thus ^ holds for t. This proves 
the induction step and thus the result holds. 

Lemma 3 Assume that (i) Assumption\l\holds and that 52S,„ax + 
"^ss^ox < '^;(ii)inAlgorithm\l\wesetaa = Bi = C^XmSmaxulbs' 

a,max, I.e. 

Then, 
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and (Hi) all additions occur before a finite time, t 

Nt = iVt„,„„, > ta,max. LctN, ^ N 

limt^oo Pr{Tt+r = Nt+-r = A^*, Vr > 0) = 1 



Proof: Since (i) and (ii) hold, Lemma[2]holds. For any i £ At, 
xt,tmp,i ~ 0. Thus, ([8} implies that {xt,i — f3t,i)'^ < Bi and so 
\Pt.^\ > \xt,,] - VB^. Thus, if \xt,,\ > \/Br+ yZfl = 2/B7, 
then f]^ i > eta, i.e. i € At. In other words, Pr{{i G Atlx^ ^ > 
4_Bi}) — 1. The same argument applies even if we consider all 
i e At. Thus, Pr({At C At}\{xl, > 4Bi Vi G At}) = 1. 

But from ^ and At C At. Thus, if xli > 4Bi, Vi G 



Ai, At = At and so T = T^i U At 



A^t. 



Thus, Pr{Tt = 

Nt = N, 



a.max 



Nt\{xij > 4Bi Vi G At}) = 1. Now, > t 
Thus for t > ta,max, Tt = A^, implies that At+i = (p. This implies 
that At = (/> and so Tt+i =Tt = N.^. Thus, T = N, implies that 
Tt+k = A., Vfc > 0. Thus, for all t > ta,max, 

Pr(Tt+r = N,\/r> 0|{x?,, > 4Bi Vi G At}) = 1 (10) 



Now, Xt^i ~ A/'(0, {t — ti)asys) where ti is the time at which ele- 
ment i got added. Note that ti < ta.max- Thus, 



ta.max 



(11) 



where Q is the Gaussian Q-function. Combining dlOl l, i ll It and using 
the fact that the different xt,i's are independent. 



Pr{Tt+r = Af, Vr > 0) > 2Q( 



4Bi 



(12) 



Thus for any e > 0, Pr{Tt+r = A^., Vr > 0) > 1 - e if t > 
ta.mai, + , = [ *^\i/s — 1, whcrc [.] is the 



,)1/S„ 
2 



greatest integer function. Thus the claim follows. ■ 

Combining Lemma[3]with Lemma[T]we get the final result. 

Theorem 2 Assume that ( i) Assumption\l\holds and that 52Smaii + 
^aSmax < 'i;{ii)inAlgorithm\l\wesetaa — Bi = ClX^Smaxiylbs. 
and (Hi) all additions occur before a finite time, ta,max, i-e. 
Nt = Ntc,^m.axi > ta,max- Consider KF-CS without the 
deletion step, i.e. with oi^ = 0, and with the step \3a\ ( KF-CS with 
final LS) replacing step\3\ifTt 7^ Tt~\. Then, dt = Xt.cAKF — Xt 
converges to zero in probability, i.e. the KF-CS estimate converges 
to the Genie-Aided KF estimate in probability, as t ^ 00. 
Also, the LS-CS estimate converges to the Genie-Aided LS estimate, 
in probability, as t ^ ca (this follows directly from Lemma\3}. 

The assumption S2s„zaw + ^ss^am < 1 is just stronger incoherency 
requirement on A than Assumption [T] The assumption of all addi- 
tions occurring before a finite time is a valid one for problems where 
the system is initially in its transient state (nonstationary), but later 
stabilizes to a stationary state. Alternatively, the above theorem can 
be applied to claim that the KF-CS (or LS-CS) estimate stabilizes 
to within a small error the GA-KF (or GA-LS) estimate, if additions 
occur slowly enough, i.e. if the delay between two addition times is 
long enough to allow it to stabilize (8). 

5. SIMULATION RESULTS 

Lemma |2] says that if the addition threshold was set high enough 
(Ua = Bi where Bi is the CS error upper bound), then there would 
be no false additions. But if we set the addition threshold very high, 
then for the initial time instants, the KF-CS estimation error would 
be large and it will do worse than simple CS. Thus, in practice, we set 
aa lower, but we implement the false addition detection and removal 
scheme described in Sec. 12.1.11 We evaluated its performance using 
the following set of simulations. We simulated a time sequence of 
sparse ?n=256 length signals, zt — xt which follow ^ with a^y^ = 
1 and nonzero sets, Nt^i C Nt, Wt satisfying Nt ^ Ni^t < 10, 
Nt = iVio,V10 < t < 20, Nt = iV2o,V20 < t < 30, iVf = 
A^30,V30 < t < 100 and |iVi| = 8, |Ario| = 12, |iV2o| = 16, 
|iV3o| = 20. Thus S,nax = 20. The set iVi and all the additions 
were generated uniformly at random from the remaining elements 
out of [1 : ?n]. The measurement matrix, A = H was simulated as 
in 12 by generating nx m i.i.d. Gaussian entries (with n = 72) and 
normalizing each column of the res ulting matrix. The observation 
noise variance was 

f^obs = ((l/3)^16/n)^ (this is taken from |2|) 
and we simulated Gaussian noise (not truncated). 
We implemented KF-CS with A™ 



^2 log2 m = 4, Oa = 
fc = 5, fc' = 3. Since the observation 
noise was not truncated, occasionally the addition step resulted in a 
very large number of false additions, which made A'j,^ Art singular 



9o-o6s, a/e = 2n, a, = al^s 




Fig. 1. MSE plots comparison. Fig. |l(a)| Large KF-CS error occurs 
at and after the new addition times, t = 10, 20, 30. But once the 
addition is detected, the error gradually reduces to that of GA-KF 
(or slightly higher). The error of simple CS (labeled as CS) is much 
larger (max value 45). Fig. |l(b)| Simple CS error beyond t = 50 
when St > 26 than is much larger in the Fig. |l(a)|(max value 425). 



(or almost singular) resulting in large errors at all future t. To prevent 
this, we set a maximum value for the number of allowed additions: 
we allowed at most (1.25n/ logj m) largest magnitude coefficient 
estimates larger than Oa to be added. Also, typically an addition 



took 2-3 time instants to get detected. Thus we set a~ 



i'^init is used instead of alyg the first time a new coefficient gets 
added). We simulated the above system 100 times and compared the 
MSE of KF-CS with that of GA-KF and of simple CS (followed by 
thresholding and least squares estimation as in Gauss-Dantzig l2l). 

In a second set of simulations, shown in Fig. |l(b)[ we started 
with 5i = 8 and for 10 < t < 50, we added 2 new elements every 5 
time units. Thus Smax = 26 = 5i , V t > 50. Note 26 > n/3 = 24, 
i-e. SzSmax cannot be smaller than 1. 

6. DISCUSSION AND ONGOING WORK 

In this work, we introduced Least Squares CS and analyzed why 
CS on the LS error in the observation will have lower error than CS 
on the raw observations (simple CS), when sparsity patterns change 
slowly enough. We also showed that if all additions occur before a 
finite time, if the addition threshold is set high enough, if UUP holds 
for Smax, and if the noise has bounded support, KF-CS (or LS-CS), 
converge to the genie-aided KF (or LS) in probability. In ongoing 
work, we are working on relaxing the first three assumptions used 
in the above result. We are also working on developing KF-CS for 
real-time dynamic MR imaging (6|. 
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